Ultrafast Cardiac Imaging Using Deep Learning for Speckle-Tracking Echocardiography.

High-quality ultrafast ultrasound imaging is based on coherent compounding from multiple transmissions of plane waves (PW) or diverging waves (DW). However, compounding results in reduced frame rate, as well as destructive interferences from high-velocity tissue motion if motion compensation (MoCo) is not considered. While many studies have recently shown the interest of deep learning for the reconstruction of high-quality static images from PW or DW, its ability to achieve such performance while maintaining the capability of tracking cardiac motion has yet to be assessed. In this article, we addressed such issue by deploying a complex-weighted convolutional neural network (CNN) for image reconstruction and a state-of-the-art speckle-tracking method. The evaluation of this approach was first performed by designing an adapted simulation framework, which provides specific reference data, i.e., high-quality, motion artifact-free cardiac images. The obtained results showed that, while using only three DWs as input, the CNN-based approach yielded an image quality and a motion accuracy equivalent to those obtained by compounding 31 DWs free of motion artifacts. The performance was then further evaluated on nonsimulated, experimental in vitro data, using a spinning disk phantom. This experiment demonstrated that our approach yielded high-quality image reconstruction and motion estimation, under a large range of velocities and outperforms a state-of-the-art MoCo-based approach at high velocities. Our method was finally assessed on in vivo datasets and showed consistent improvement in image quality and motion estimation compared to standard compounding. This demonstrates the feasibility and effectiveness of deep learning reconstruction for ultrafast speckle-tracking echocardiography.

Complex Convolutional Neural Networks for Ultrafast Ultrasound Imaging Reconstruction From In-Phase/Quadrature Signal.

Ultrafast ultrasound imaging remains an active area of interest in the ultrasound community due to its ultrahigh frame rates. Recently, a wide variety of studies based on deep learning have sought to improve ultrafast ultrasound imaging. Most of these approaches have been performed on radio frequency (RF) signals. However, in- phase/quadrature (I/Q) digital beamformers are now widely used as low-cost strategies. In this work, we used complex convolutional neural networks for reconstruction of ultrasound images from I/Q signals. We recently described a convolutional neural network architecture called ID-Net, which exploited an inception layer designed for reconstruction of RF diverging-wave ultrasound images. In the present study, we derive the complex equivalent of this network, i.e., complex-valued inception for diverging-wave network (CID-Net) that operates on I/Q data. We provide experimental evidence that CID-Net provides the same image quality as that obtained from RF-trained convolutional neural networks, i.e., using only three I/Q images, CID-Net produces high-quality images that can compete with those obtained by coherently compounding 31 RF images. Moreover, we show that CID-Net outperforms the straightforward architecture that consists of processing real and imaginary parts of the I/Q signal separately, which thereby indicates the importance of consistently processing the I/Q signals using a network that exploits the complex nature of such signals.

Reconstruction for Diverging-Wave Imaging Using Deep Convolutional Neural Networks.

In recent years, diverging wave (DW) ultrasound imaging has become a very promising methodology for cardiovascular imaging due to its high temporal resolution. However, if they are limited in number, DW transmits provide lower image quality compared with classical focused schemes. A conventional reconstruction approach consists in summing series of ultrasound signals coherently, at the expense of frame rate, data volume, and computation time. To deal with this limitation, we propose a convolutional neural network (CNN) architecture, Inception for DW Network (IDNet), for high-quality reconstruction of DW ultrasound images using a small number of transmissions. In order to cope with the specificities induced by the sectorial geometry associated with DW imaging, we adopted the inception model composed of the concatenation of multiscale convolution kernels. Incorporating inception modules aims at capturing different image features with multiscale receptive fields. A mapping between low-quality images and corresponding high-quality compounded reconstruction was learned by training the network using in vitro and in vivo samples. The performance of the proposed approach was evaluated in terms of contrast ratio (CR), contrast-to-noise ratio (CNR), and lateral resolution (LR), and compared with standard compounding method and conventional CNN methods. The results demonstrated that our method could produce high-quality images using only 3 DWs, yielding an image quality equivalent to that obtained with compounding of 31 DWs and outperforming more conventional CNN architectures in terms of complexity, inference time, and image quality.

Comparison Between Multiline Transmission and Diverging Wave Imaging: Assessment of Image Quality and Motion Estimation Accuracy.

High frame rate imaging is particularly important in echocardiography for better assessment of the cardiac function. Several studies showed that diverging wave imaging (DWI) and multiline transmission (MLT) are promising methods for achieving a high temporal resolution. The aim of this study was to compare MLT and compounded motion compensation (MoCo) DWI for the same transmitted power, same frame rates [image quality and speckle tracking echocardiography (STE) assessment], and same packet size [tissue Doppler imaging (TDI) assessment]. Our results on static images showed that MLT outperforms DW in terms of resolution (by 30% on average). However, in terms of contrast, MLT outperforms DW only for the depth of 11 cm (by 40% on average), the result being reversed at a depth of 4 cm (by 27% on average). In vitro results on a spinning phantom at nine different velocities showed that similar STE axial errors (up to 2.3% difference in median errors and up to 2.1% difference in the interquartile ranges) are obtained with both ultrafast methods. On the other hand, the median lateral STE estimates were up to 13% more accurate with DW than with MLT. On the contrary, the accuracy of TDI was only up to ~3% better with MLT, but the achievable DW Doppler frame rate was up to 20 times higher. However, our overall results showed that the choice of one method relative to the other is therefore dependent on the application. More precisely, in terms of image quality, DW is more suitable for imaging structures at low depths, while MLT can provide an improved image quality at the focal point that can be placed at higher depths. In terms of motion estimation, DW is more suitable for color Doppler-related applications, while MLT could be used to estimate velocities along selected lines of the image.

heartBEATS: A hybrid energy approach for real-time B-spline explicit active tracking of surfaces.

In this manuscript a novel method is presented for left ventricle (LV) tracking in three-dimensional ultrasound data using a hybrid approach combining segmentation and tracking-based clues. This is accomplished by coupling an affine motion model to an existing LV segmentation framework and introducing an energy term that penalizes the deviation to the affine motion estimated using a global Lucas-Kanade algorithm. The hybrid nature of the proposed solution can be seen as using the estimated affine motion to enhance the temporal coherence of the segmented surfaces, by enforcing the tracking of consistent patterns, while the underlying segmentation algorithm allows to locally refine the estimated global motion. The proposed method was tested on a dataset composed of 24 4D ultrasound sequences from both healthy volunteers and diseased patients. The proposed hybrid tracking platform offers a competitive solution for fast assessment of relevant LV volumetric indices, by combining the robustness of affine motion tracking with the low computational burden of the underlying segmentation algorithm.

High-Quality Plane Wave Compounding Using Convolutional Neural Networks.

Single plane wave (PW) imaging produces ultrasound images of poor quality at high frame rates (ultrafast). High-quality PW imaging usually relies on the coherent compounding of several successive steered emissions (typically more than ten), which in turn results in a decreased frame rate. We propose a new strategy to reduce the number of emitted PWs by learning a compounding operation from data, i.e., by training a convolutional neural network to reconstruct high-quality images using a small number of transmissions. We present experimental evidence that this approach is promising, as we were able to produce high-quality images from only three PWs, competing in terms of contrast ratio and lateral resolution with the standard compounding of 31 PWs ( 10× speedup factor).

A Sparse Reconstruction Framework for Fourier-Based Plane-Wave Imaging.

Ultrafast imaging based on plane-wave (PW) insonification is an active area of research due to its capability of reaching high frame rates. Among PW imaging methods, Fourier-based approaches have demonstrated to be competitive compared with traditional delay and sum methods. Motivated by the success of compressed sensing techniques in other Fourier imaging modalities, like magnetic resonance imaging, we propose a new sparse regularization framework to reconstruct highquality ultrasound (US) images. The framework takes advantage of both the ability to formulate the imaging inverse problem in the Fourier domain and the sparsity of US images in a sparsifying domain. We show, by means of simulations, in vitro and in vivo data, that the proposed framework significantly reduces image artifacts, i.e., measurement noise and sidelobes, compared with classical methods, leading to an increase of the image quality.

Extension of Fourier-Based Techniques for Ultrafast Imaging in Ultrasound With Diverging Waves.

Ultrafast ultrasound imaging has become an intensive area of research thanks to its capability in reaching high frame rates. In this paper, we propose a scheme that allows the extension of the current Fourier-based techniques derived for planar acquisition to the reconstruction of sectorial scan with wide angle using diverging waves. The flexibility of the proposed formulation was assessed through two different Fourier-based techniques. The performance of the derived approaches was evaluated in terms of resolution and contrast from both simulations and in vitro experiments. The comparisons of the current state-of-the-art method with the conventional delay-and-sum technique illustrated the potential of the derived methods for producing competitive results with lower computational complexity.

Left-Atrial Segmentation From 3-D Ultrasound Using B-Spline Explicit Active Surfaces With Scale Uncoupling.

Segmentation of the left atrium (LA) of the heart allows quantification of LA volume dynamics which can give insight into cardiac function. However, very little attention has been given to LA segmentation from three-dimensional (3-D) ultrasound (US), most efforts being focused on the segmentation of the left ventricle (LV). The B-spline explicit active surfaces (BEAS) framework has been shown to be a very robust and efficient methodology to perform LV segmentation. In this study, we propose an extension of the BEAS framework, introducing B-splines with uncoupled scaling. This formulation improves the shape support for less regular and more variable structures, by giving independent control over smoothness and number of control points. Semiautomatic segmentation of the LA endocardium using this framework was tested in a setup requiring little user input, on 20 volumetric sequences of echocardiographic data from healthy subjects. The segmentation results were evaluated against manual reference delineations of the LA. Relevant LA morphological and functional parameters were derived from the segmented surfaces, in order to assess the performance of the proposed method on its clinical usage. The results showed that the modified BEAS framework is capable of accurate semiautomatic LA segmentation in 3-D transthoracic US, providing reliable quantification of the LA morphology and function.

Compressed Sensing Doppler Ultrasound Reconstruction Using Block Sparse Bayesian Learning.

In this paper we propose a framework for using duplex Doppler ultrasound systems. These type of systems need to interleave the acquisition and display of a B-mode image and of the pulsed Doppler spectrogram. In a recent study (Richy , 2013), we have shown that compressed sensing-based reconstruction of Doppler signal allowed reducing the number of Doppler emissions and yielded better results than traditional interpolation and at least equivalent or even better depending on the configuration than the study estimating the signal from sparse data sets given in Jensen, 2006. We propose here to improve over this study by using a novel framework for randomly interleaving Doppler and US emissions. The proposed method reconstructs the Doppler signal segment by segment using a block sparse Bayesian learning (BSBL) algorithm based CS reconstruction. The interest of using such framework in the context of duplex Doppler is linked to the unique ability of BSBL to exploit block-correlated signals and to recover non-sparse signals. The performance of the technique is evaluated from simulated data as well as experimental in vivo data and compared to the recent results in Richy , 2013.

Compressed Sensing Reconstruction of 3D Ultrasound Data Using Dictionary Learning and Line-Wise Subsampling.

In this paper we present a compressed sensing (CS) method adapted to 3D ultrasound imaging (US). In contrast to previous work, we propose a new approach based on the use of learned overcomplete dictionaries that allow for much sparser representations of the signals since they are optimized for a particular class of images such as US images. In this study, the dictionary was learned using the K-SVD algorithm and CS reconstruction was performed on the non-log envelope data by removing 20% to 80% of the original data. Using numerically simulated images, we evaluate the influence of the training parameters and of the sampling strategy. The latter is done by comparing the two most common sampling patterns, i.e., point-wise and line-wise random patterns. The results show in particular that line-wise sampling yields an accuracy comparable to the conventional point-wise sampling. This indicates that CS acquisition of 3D data is feasible in a relatively simple setting, and thus offers the perspective of increasing the frame rate by skipping the acquisition of RF lines. Next, we evaluated this approach on US volumes of several ex vivo and in vivo organs. We first show that the learned dictionary approach yields better performances than conventional fixed transforms such as Fourier or discrete cosine. Finally, we investigate the generality of the learned dictionary approach and show that it is possible to build a general dictionary allowing to reliably reconstruct different volumes of different ex vivo or in vivo organs.

Back-propagation beamformer design for motion estimation in echocardiography.

Transverse oscillation (TO) techniques have shown their potential for improving the accuracy of local motion estimation in the transverse direction (i.e., the direction perpendicular to the beam axis). The conventional design of TOs in linear geometry, which is based on the Fraunhofer approximation, relates point spread function (PSF) and apodization function through a Fourier transform. Motivated by the adaptation of TOs in echocardiography, we propose a specific beamforming approach based on back-propagation (BP) to build TOs in sector-shaped geometry. Numerical simulations and experimental data give a comparison between proposed and conventional beamforming for TOs. The accuracy is first quantified by comparing the generated and theoretical PSF using the root mean square error (RMSE) and shows that BP-based beamforming approximates the desired TOs more closely than the conventional approach. Motion estimation is then evaluated. The axial and lateral displacements are within the range [0-0.6] mm and [0°-6.4°], respectively, which correspond to 0.8 times the axial (0.73 mm) and lateral (8°) wavelengths. The result shows that the proposed method yields a clear improvement for lateral displacements, by reducing the error by 28.6% compared with Fourier transform-based beamforming, while maintaining the same error for axial motion estimation. Experimental measurements are discussed to complete this study and confirm that BP-based beamforming leads to better controlled TO images than conventional Fourier-based beamforming.

OntoVIP: an ontology for the annotation of object models used for medical image simulation.

This paper describes the creation of a comprehensive conceptualization of object models used in medical image simulation, suitable for major imaging modalities and simulators. The goal is to create an application ontology that can be used to annotate the models in a repository integrated in the Virtual Imaging Platform (VIP), to facilitate their sharing and reuse. Annotations make the anatomical, physiological and pathophysiological content of the object models explicit. In such an interdisciplinary context we chose to rely on a common integration framework provided by a foundational ontology, that facilitates the consistent integration of the various modules extracted from several existing ontologies, i.e. FMA, PATO, MPATH, RadLex and ChEBI. Emphasis is put on methodology for achieving this extraction and integration. The most salient aspects of the ontology are presented, especially the organization in model layers, as well as its use to browse and query the model repository.

Fast automatic myocardial segmentation in 4D cine CMR datasets.

A novel automatic 3D+time left ventricle (LV) segmentation framework is proposed for cardiac magnetic resonance (CMR) datasets. The proposed framework consists of three conceptual blocks to delineate both endo and epicardial contours throughout the cardiac cycle: (1) an automatic 2D mid-ventricular initialization and segmentation; (2) an automatic stack initialization followed by a 3D segmentation at the end-diastolic phase; and (3) a tracking procedure. Hereto, we propose to adapt the recent B-spline Explicit Active Surfaces (BEAS) framework to the properties of CMR images by integrating dedicated energy terms. Moreover, we extend the coupled BEAS formalism towards its application in 3D MR data by adapting it to a cylindrical space suited to deal with the topology of the image data. Furthermore, a fast stack initialization method is presented for efficient initialization and to enforce consistent cylindrical topology. Finally, we make use of an anatomically constrained optical flow method for temporal tracking of the LV surface. The proposed framework has been validated on 45 CMR datasets taken from the 2009 MICCAI LV segmentation challenge. Results show the robustness, efficiency and competitiveness of the proposed method both in terms of accuracy and computational load.

A new technique for the estimation of cardiac motion in echocardiography based on transverse oscillations: a preliminary evaluation in silico and a feasibility demonstration in vivo.

Quantification of regional myocardial motion and deformation from cardiac ultrasound is fostering considerable research efforts. Despite the tremendous improvements done in the field, all existing approaches still face a common limitation which is intrinsically connected with the formation of the ultrasound images. Specifically, the reduced lateral resolution and the absence of phase information in the lateral direction highly limit the accuracy in the computation of lateral displacements. In this context, this paper introduces a novel setup for the estimation of cardiac motion with ultrasound. The framework includes an unconventional beamforming technique and a dedicated motion estimation algorithm. The beamformer aims at introducing phase information in the lateral direction by producing transverse oscillations. The estimator directly exploits the phase information in the two directions by decomposing the image into two 2-D single-orthant analytic signals. An in silico evaluation of the proposed framework is presented on five ultra-realistic simulated echocardiographic sequences, where the proposed motion estimator is contrasted against other two phase-based solutions exploiting the presence of transverse oscillations and against block-matching on standard images. An implementation of the new beamforming strategy on a research ultrasound platform is also shown along with a preliminary in vivo evaluation on one healthy subject.

Real-time 3D interactive segmentation of echocardiographic data through user-based deformation of B-spline explicit active surfaces.

Image segmentation is an ubiquitous task in medical image analysis, which is required to estimate morphological or functional properties of given anatomical targets. While automatic processing is highly desirable, image segmentation remains to date a supervised process in daily clinical practice. Indeed, challenging data often requires user interaction to capture the required level of anatomical detail. To optimize the analysis of 3D images, the user should be able to efficiently interact with the result of any segmentation algorithm to correct any possible disagreement. Building on a previously developed real-time 3D segmentation algorithm, we propose in the present work an extension towards an interactive application where user information can be used online to steer the segmentation result. This enables a synergistic collaboration between the operator and the underlying segmentation algorithm, thus contributing to higher segmentation accuracy, while keeping total analysis time competitive. To this end, we formalize the user interaction paradigm using a geometrical approach, where the user input is mapped to a non-cartesian space while this information is used to drive the boundary towards the position provided by the user. Additionally, we propose a shape regularization term which improves the interaction with the segmented surface, thereby making the interactive segmentation process less cumbersome. The resulting algorithm offers competitive performance both in terms of segmentation accuracy, as well as in terms of total analysis time. This contributes to a more efficient use of the existing segmentation tools in daily clinical practice. Furthermore, it compares favorably to state-of-the-art interactive segmentation software based on a 3D livewire-based algorithm.

Blood velocity estimation using compressive sensing.

Duplex ultrasonography is a mode of medical ultrasonography that allows one to visualize, at the same time, the inner structure of the body (B-mode) and the blood flow at a particular point in the body (Doppler mode). This mode requires a strategy for alternating B-mode and flow emissions. Traditional strategies either halve the maximum measurable velocity or introduce gaps in the flow data. The objective of this article is to propose a completely original method based on compressive sensing for reconstructing the Doppler signal segment by segment. Our approach is based on randomly alternating B-mode and flow emissions. The influence of the different parameters on the reconstruction quality is studied in detail. The technique is evaluated and its feasability is validated in simulation and from experimental in vivo data. It is also compared to the only method from the literature, proposed by Jensen, that reconstructs blood velocity estimates from sparse data sets.

A virtual imaging platform for multi-modality medical image simulation.

This paper presents the Virtual Imaging Platform (VIP), a platform accessible at http://vip.creatis.insa-lyon.fr to facilitate the sharing of object models and medical image simulators, and to provide access to distributed computing and storage resources. A complete overview is presented, describing the ontologies designed to share models in a common repository, the workflow template used to integrate simulators, and the tools and strategies used to exploit computing and storage resources. Simulation results obtained in four image modalities and with different models show that VIP is versatile and robust enough to support large simulations. The platform currently has 200 registered users who consumed 33 years of CPU time in 2011.

Pre-beamformed RF signal reconstruction in medical ultrasound using compressive sensing.

Compressive sensing (CS) theory makes it possible - under certain assumptions - to recover a signal or an image sampled below the Nyquist sampling limit. In medical ultrasound imaging, CS could allow lowering the amount of acquired data needed to reconstruct the echographic image. CS thus offers the perspective of speeding up echographic acquisitions and could have many applications, e.g. triplex acquisitions for CFM/B-mode/Doppler imaging, high-frame-rate echocardiography, 3D imaging using matrix probes, etc. The objective of this paper is to study the feasibility of CS for the reconstruction of channel RF data, i.e. the 2D set of raw RF lines gathered at the receive elements. Successful application of CS implies selecting a representation basis where the data to be reconstructed have a sparse expansion. Because they consist mainly in warped oscillatory patterns, channel RF data do not easily lend themselves to a sparse representation and thus represent a specific challenge. Within this perspective, we propose to perform and assess CS reconstruction of channel RF data using the recently introduced wave atoms [1] representation, which exhibit advantageous properties for sparsely representing such oscillatory patterns. Reconstructions obtained using wave atoms are compared with the reconstruction performed with two conventional representation bases, namely Fourier and Daubechies wavelets. The first experiment was conducted on simulated channel RF data acquired from a numerical cyst phantom. The quality of the reconstructions was quantified through the mean absolute error at varying subsampling rates by removing 50-90% of the original samples. The results obtained for channel RF data reconstruction yield error ranges of [0.6-3.0]×10(-2), [0.2-2.6]×10(-2), [0.1-1.5]×10(-2), for wavelets, Fourier and wave atoms respectively. The error ranges observed for the associated beamformed log-envelope images are [2.4-20.6]dB, [1.1-12.2]dB, and [0.5-8.8dB] using wavelets, Fourier, and wave atoms, respectively. These results thus show the superiority of the wave atom representation and the feasibility of CS for the reconstruction of US RF data. The second experiment aimed at showing the experimental feasibility of the method proposed using a data set acquired by imaging a general-purpose phantom (CIRS Model 054GS) using an Ultrasonix MDP scanner. The reconstruction was performed by removing 80% of the initial samples and using wave atoms. The reconstructed image was found to reliably preserve the speckle structure and was associated with an error of 5.5dB.

Quantification of left ventricular volume and global function using a fast automated segmentation tool: validation in a clinical setting.

Real-time 3D echocardiography (RT3DE) has already been shown to be an accurate tool for left ventricular (LV) volume assessment. However, LV border detection in RT3DE remains a time-consuming task jeopardizing the application of this modality in routine practice. We have recently developed a 3D automated segmentation framework (BEAS) able to capture the LV morphology in real-time. The goal of this study was to assess the accuracy of this approach in extracting volumetric parameters in a clinical setting. 24 RT3DE exams were acquired in a group of healthy volunteers (# = 5) and diseased patients (# = 19), with LV volume/function within a range typically measured in a clinical setting. End-diastolic and end-systolic volumes (EDV, ESV) were manually contoured by 3 expert sonographers from which the stroke volume and ejection fraction (SV, EF) were calculated. The values extracted with BEAS were compared to the average of the 3 experts measurements using correlation and Bland-Altman statistics. Linear regression analysis showed a strong correlation between the automated algorithm and the reference values (R = 0.963, 0.947, 0.944 and 0.853 for EDV, ESV, SV and EF respectively). Bland-Altman analysis revealed a bias (limits of agreement) of 2.59 (-25.39, 30.57) ml, -2.11 (-24.91, 20.69) ml, 4.70 (12.93, 22.34) ml and 3.45 (-8.96, 15.87) %, for EDV, ESV, SV and EF respectively. Total analysis time using BEAS was 30.7 ± 7.5 s. BEAS allows for a fast and accurate quantification of 3D cardiac volumes and global function with minimal user input. It may therefore contribute to the integration of 3D echocardiography in routine clinical practice.